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%> * Computer Assisted Problem Sol ving 

1 

Abstract 

•The cpmpu/ter assisted problem solving system (CAPS) administered "homework" 
Rroblem-sets designed to develop students' computational , estimation, a'nd 
p^rocedi/i^al ski Ms rei"ate.d to important concepts of' an introductory statis- 

■ / - \ ■ ^. • ; . . ^ 

trcs course. CAPS generated unique data, Judged student performance, 
provided hints ^bout problem solving strategies, and stored "relevant per- 

^ ' ' ' 

^ form'ance data. A PDP-IO computer, with hardcopy and CRT terminals, was 

• " ' / 

programmed to' inte^ract with students Jn an instructional mode* Courseware 

• '''•/ \ ■ • /i 

was coded iri rnteractive Fortran and packaged so that additional items cart 

^ \ ' • ' ■ • ' / ' 

^be added easily ta tne existing problem sets, Aftfer three semesters of 
implem^ntat ion,^ stude)it reaction to Is generally favorable. ^- 
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Computer Assisted Problem J^olv Ing in an 
Intrc/ductory^Stat ist ics Course 
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The computer assi^ed problem solv ingjsystem (CAPS) vias designed to 
"hel p Stat i!^t Ics students acquire computat^yon s'kills and exjje^ience in' 
solving a set of rather traditional statf/sticaV exercises/ CAPS is but one. 

- / ^ . * ' 

component of an instructional systems approach .used to teach Ian Jntroductory 
statistics course/and, as such, needs to be-describedc within the context pf 
the course. 

Organization of the Course 



whQ 



cs course s rnce 



The course serves a large number of- students wjth a wide range of 
abilities, professional asfji rations, and special i t ies. Some students 
enroll In the course, for exam|ple> bm/e not had a mathi^ati 
high school, while .others have recently completed college level courses. 
Some ef;iter the course with a crippling ^fear of statistics, while others\ 
approach it^ith breezy confidence. • ^ ' ■ " * 




'In response to the'se ne^ds, the logistics'bf the traditiorlal lecture- 

' /' * « ' '- ^ ^ 

discussion course was modified and some new components were added. The tvyo ' 

'«/''*/' - ' 

hour^s per Week lecture-discussion sessions, and the primary text (GTass*^ 

, ■ / ^ ~ " ' ■ ' ^ • . • 

Stanley, /i970) were retafried, The lectures, were "tightened up" sonewhal;, ' 
and the /d isTcusslon typicajly was used -to clarify points ra*t|ier than .'*enr Tch**^ 

' i / . I ' f \ * ' 

the content. In preparatiorv' for the class sect ion fol lowtrig lect.ures, ^about 

/ . . • ' J ^ ' . 

four days later, all students were requ i red* to take a quiz at thel;* vconv.enlence 

on the PLATO IV CAI system-\(Paul son,M 978) . If the studen.t- obtained a , score . 

of /B5"100% correct on| a 10-20 item quiz, attendance at, the next ^Tession w^s7 
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— -. • -.3 

ustjaHy optional; a score of 70-85% correct meant that the student* wcis ' 
St rongflT^ncou raged tx) attend; and *a score^belovy 70% required the student 
to attend. This class session was ^usually a prdblem-or iented, irvformal 
session. ^ • ^ 

\ ' ^ ■ _ 'r ' ^ . . ' 

"•y To assist the student 'who had difficulty reading /the primary text and 

needed more examples of st;at'jstics principles, two of the .i nstructors pre- 

pared a progrannmed text and workbook (Armbruster S/Dastrup, 1976) . Finally, 

* » » • 

al.l students were required to /solvfe 20 problems the CAPS system. ^This 

requi rement* replaced the typical homework probfem sets. The proj^lems were 

designed to test students* abilities to seleQt, compute, and estinoate^ 

statistics with an emphasis on applying them. The CAPS items compS^ejo^ted 

the PLATO quizzes io that- the quizzes tested f)rinciples and concept%,;^H^e 

the CAPS items stressed computational exercises relate4 to the co^ce 

ft 

How CAPS Works '' 
: ^ ' , 

The CAPS procedure -started for a student whert she' took her instri 

•sheet ^d signe(^ on at a DEC .writer or INFOTON CRT terminal connected 

PDP-lO/ system: In an. interact ive mode, the student received informatf^;; 

how to use the S-Vstem, and"^ a combined index and IProgress "keport paQe c^jli^lS 

tf^e problems. The student first selected a problem to solve which, was -'||;^, 

printed", after which she signed off the system and attempted to solve ipj^.|^^ 

Later she signed back onto the system' and recorded her artswer. 'frifMC^ 
/ ' ^ * ' * ' *' 

computer judged the answer to be correct (within'a tolerance described in \ -^.|' 

/ ' . .7 ■ , ■ : • . V'^^ 

the probVem) the student was returned to the index page; if it was incorrect, 
9 problem solving' hint was provided and the student worked until the problem 
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was sdlyed correctly. or the student received four hints. This [procedure 

/ . r * . 

is il'lustrated. in Figure 1. 




\ 

\ 



I Insert figure } about here 
' . ^ 

Prob'lem Solving Feedback to Students ' , - 

iT ' " — ^ 

The CAPS items ^were class ified into two types: (l) estimation probl^fims 

' . ..^ * * 4* • / • ' * ' ' 

requiring approxii^r^tlon of a statistic from a graphic representation of 

• ■ ■ '\ ' . * U - ^ : . 'x 

data and iZ) computation problems for which there were preferred strategi^es 

. ^ * * 

of solution. .The basi-c \n^ri/ctiorkar design of JCAPS wa^ to incorporate 

• . */\ ' iki . * 

feed^acT< strategies fpr each of tKese two- types pf problems! • i 

\ / . ' . . ' ' ' 

.For corT>putat^.onal problem^, four levels of problem solving strategies 

(hints) were developed. The first hint was simply a restatement of the 

problem. -The excess Verba^ge. was .stripped away and the Ltem was organized 

under tfie labels "Givefi*' and "Problem.*' Hint Z was'a- strategy for working 

the problem listed i.n a series of steps. The strategy fs, similar to what 

a tutor might use when working with a>^studertt. .Hint 3'proyided a solutiopy 

^to a's'imilai^ probleril, including intermediate solutions congruent wlt^ the ^ 

steps .provided in Hint 2y Hint 4 .presented the solution, with intermediate 

steps, to .the- given problem,^ 

To make the hints more me aningful and to diminish th€**tendency of some 

stude^its *to share answers, ^the. computer generated unique problems for each 

student. ' Thd basic item format remained the same but the data base changed. 

Eadh problem, then, \s a one i tern' doma i n ' referenced test or an item form, ^ 

as described b^^Hively et al . (1973)- In this way, students could call for 

\ • , • • N r ' , r ' . ^ -r ' • . ■ * ' 

as mac^y , Hint 3's aSothey-^Wished ^and-see /the pr^*Jem"solut ion with different 

data each time, * ^ ^ 
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* . Two of ^the problems were designed, to sharpen a student's abTlity to 

•/''/' " ^ ' . ^ 

estimate statistTcs from a large array of data. In 'one probl ^m-," ten unique 

histograms with afiproximately 60 entries each were displayed,., and- the^stddent 

was required to estimate the mean and standard deviation 6f each with 

' ' . * » • / ' 

tolerances of approx imatel y 3% and 10^ respectively. At least itf correct 

* estim^es of the 20 possibilities were recJUired before the probfem was judged' 

to be completed successf uTly. About half of the ten histogramsf/ represented 

» distributions with approximate nprmaj distributions ^vhi le the jremaining pnes 

/were skewed and/or bimodal.' 

* », ' > * ' » 

' ' In another problem, ten. unique scatterplots with ab3\jt 25 eVitries each 

^ ■ • " I '-^ 

^•were. displayed, and the students were required to estimate ths Pearson 

*' • * - ' "j ' 

product-moment correlation coefficient within ^a tolerance of .05. At least 

seven correct es^lm^tes of 'the posijible 10 were required to cAmalete the ^ 

problem s)>rt:essf ul lY- The^ coef f I cie;nts ranged approximately from -1 to +K 

* ' I • * 

. ^ On^^ other type of problem tested procedural Skills. The atudents Were , 

' i ' y - 1 ' 

Required to locate Chi, F; and T statistics in a table With .prescribed |irob- 

ability and degree of freedom parameters. Twelve items 'were presented and 

the st.ud'^^g^ts w^re required to answer them aTl cot^Vectl y . x \ 

The remaining problems dealt with (1) computation of descriptive sta- , 

tistlcs such as mean, centime rank, and variance, (2) ident if icai ion and 




computert-to n o f an fnfc rential statist/tc, such as differences betjwe*e^the 
means of depend;ent samples^ and (5) correlation an3 est imat ion. ^ Tftfe Index 
of problems, with' abbreviated titles/, is shdvln In Figure 2. 

. • • , . Insert Figure 2 abput- here . 
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"^'../•^ Computer-Assisted* Problem Solving 

Students wer.e required to sol ve *20' proW em types during the semester 
using? no more than one hint on each befoce they could 'take the final exart. 
Problems could be selected as often as necessary to meet this requirement. 

To date,; CABS has been used fpr^two semesters with approximately 100 
students*. The remainder of the paper d i scusses, the preparatiori-and design 
of fhe software and presents evaluation-type data from students. 
Hardware - Software ' . , 

A PDP-10 time^harjng computer (Digital Equipment Corporation) was chosen' 

¥ * ' * 

to 'deliver CAPS for two reasons. 'First, another timesharing dbmputer based 
educat ion . system had already been used sucessfully for another course 

* ♦ 

management! project (Alessi et ah, 1976; Anderson et al,.^ 197^, 1975 a,b). 
We wished to^ inve§.tigate tite feasibility of instruct ional* del ivery on another 
system. Secpnd, the POP- 10 provided a FORTRAN compiler, While^ PLATO provides 
only the TUTOR language (Sherwood, 197^) which is available only on PLATO. 
We wished' to test, the feasibility of aij instruct"i€)nal system using thp more^ 
common FORTRAN langrfS^ge. ^ # 

The PDP-lO^Js a, programmer *s computer with ver/poWerful software and 

■ s . 

is correspondingly difficult to use./ We were able to use certain system . 
|eatures, however, to m^e the PDP-10 usuable by students with minimal in- 
struction. A Student needed only to be able to sign on* From there, the 
correct prog^rams are automatically executed and the student automatically . 



signed off at the end of the session. Subroutines in the program check the 
form of student responses (i .e.» alphabetic versus nurrtberic) and give the 
student meaningful feedback when incorrect, rather than the usu^l coded 
system error messages meant for programmers, 



. Co/nputer Assisted Problem Solving 

• *'*''. • 

* ' / 

* * * • 1 

The CAPS Rrograms' consist of a main program wit*h„a number of overlaying 
subroutines., "She 'subroutines^. perform the following functions: ^1 ) The 
I tern ^subrout ines generate data and display the 20' items (each jtem-/<-^ 

being a^ separate subrout itie) . The data for any item are generated by'a 
random number generator 'seeded by a function of fbe student number, item - 
number, and how many times the student has tried that item. With such a 
method of data 'gene|ratiorr5 — it— fs--pp5sible for an liT|y-uctor to obtain, th^ 
exact d«ita a particular studeot received on any 'tr>^W an item. ' (2) The 

Student inpUt subroutine i s 'respons I ble for acceptsing all student yfinputs, 

. / . ' / , , " • 

checking their fo^fm, %nd providing help if wrong. (3) Two.di fferept' logoff 

subroutines log the student off either at the stadent*s request (a single 

keypress available' at any time) or at the end of the session. (A) The 

answer judging* subroutine compares a student^ response to the correct answer 

calculated by the individijal Item subroutines, and judges it to be? correct 

or incorrect. (5) A few subroutines segregate' code that is not standard 

ANS FORTRAN,^ which' would require modification on other computers. (6) A 

* pause' subroutine stdps execution of the program to give students time to 

copy information from the terminal before it disappears. T^ost students 

worked at CRT terminals, and the item display, whicK geherally fills a 

screen, woD^ldf be partially lost when logout occurs. (7) An Information sub- 

routine provides students with a short lesson on t|te use^f the terminal and 

the CAPS system. (8) An instructor sub;-outine .provides a facilltY to Inspect 

any item as it appeared for any student on. any try^i'- 

' " ; . . 

Subroutines are overlayed because 'the entire set'of programs is too big 

• * * ^' 

to fit in central memory at one time. With overlaying, no student satesion 
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fcccupies more than 16.5 kilobytes of corfe for the main prbgram and necessary 
^!Sw^6ut^nes. The logoff and' paus^ subrout i nes are coded-in MACRO-10, PDP-10 



dsfembl\ 



language. .All others are coded in FORTfy\Ni 



The main program,, always called at login,''perf orris* a variety of functions 
It accesses an individual student's record (determined by the student in- 
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putting her/his student number and password) and updates them at logoff. 
It displays messages from the ' instructors targeted for any -individual studpnt 
or the entire class. It displays the index when appropriate, or calls the 
rjpcessary item subroutine; lastly, it accesses the logoff subroutines.' 

Student records are kept in a direct-access disc fiVe. "The student 
number given at login correjsponds to the same number record * in* the file. 
The record contains the stude'nt's password; scores, level ,df completion, 
and number of tries for each of the twenty i.tems; whether the student- has a 
message .not yet* seen, and whether the:student has seen -the introductory fin- 
.formation on the use of the terminal and CAPS. The record is read in ait 
signin and Written out both at logoff and whenever a significant activity 
is completed, s.uch 'as completing an item. The latter Js done to protect 
against the loss of data due to system or terminal failure. 
Evaluation . ^ r 



In addition! to performance data described' previously, we periodically. 
inspe<?ted the data file to tfack chronological ly the se<juence of items 

; * ^ w • ' 

selected and hints required on^trjals. The procedure is to print the data 
file every two Jweeks during the^first 12*weeks and to print it every few 
days during the last weeks of the semester as student activity increases. 
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The students also completed two questionnaires (at;, mf^-^ern^tar^-^he end 
of the semester) during the course of the semester whrcJi Mncl uded'"^"i tems on" 
a J 1 aspects of the course' including Jthe CAPS system/ 8oth,.FaII semester 

• . - / r \ ' 

and the first Spring semester quest i ona3 i res/were administered through the 

PLATO system^ used for quizzes dunii^ the semesters. 

We have dfescribed the revised version. jof CAPS which was used during the 

second semestfer of its operation. The evaluation of performance data from 

the first semester, which included student questionnaire resfthts anc| an / 
* ^ . * » 

analysis of cos-ts involved, *resul ted in several *pf09f3'Timing changes. Most 
of the changes were directed at 'student conve/iienc^' with little attention 
to cost .factors. However, the changes did result' in a HO^decpease In the 

cost pe.r completed item. - * * *\ *' 

i ' • . ' • 

Originally, the strategy for CAP'S was that a student^would select an- 

• • . ^ • J • 

item anql remain on the system while doing the calculations. If the answer 



ged 9ff the system,^ to rework the item. After & second incorrect 



was judged as incdrrect^ tfie student was shown the first hint and then log- 
answer on If 

the second hint was shown and the student was signed , off. This 'sign-in, . . 
record 'an answer, receive a' hint if wrong and ^be signed off* cycle might 

'r ' ^ ' , i ' ' • 

be^repeated four tjmes before the student could, select another Ttem. Since 
the student wa"s requi/ed to repeat the I t-em 'i f 'noore than one incorrect ^ 
response was gi^en, -it ^was'^felt ^that hints 2, 3, and h should be avaMable 
.after the second incorrect respdnse rather than require four separate s'lgn- 
Jns to i'nspect all hints. , * • ' 

Another chartge'was to require that*the students work the problem*s off 
line, pafher than see the .item-and work'it immediately,' we programm^^^t^the 
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computer to log the student off after a "two minute, pause. This Qieant that 



we bou.ld delete the calQulat%r subroutine that we had made availaBle in.an 
earlier version. We felt justif ied< in-our decision to dl scont i nue.us iTig 
the calptilator since questionnaire pesults indicated that more than '50% 6f 
the students never i^ed^the^alculatar and on*ly 10% used i t .regular ly,* 

These^changesvpl us the deletion of two items r^ated to concepts not 
taught in the^ introductory statistics course resulted in a decrease In cos't 
per item and a decrease in. computer connect, time per item cfuring the second 
semest-er of. operation*. A summarY'of costytime data for tMe fi rst • semester 
.and the first half or- the second semester is shown in Figure 3. The basic 
charges for PDP-10 u^e* fnclude: '$2.40*per connect hour, $1.50 ger kilocore 
hour, and $$0.00 per CPU hour.. In general, the average charge is abdut-$5.00 

* 4 1 • 

per hour of terminal use during * norma IV edlti/ig and/or execution of programs. 

( . ^ * > , . ' 

V ' ' . ^ ' 

Insert Figure 3 about here ^ ' • 

^ >—— — — — •- — — — — —— — -* — — — — — — — — — — — '— » V « •« 

As pne might expect, student reaction to CAPS has ranged from very 
•positive to assertati vel Y negative. The, range of reactions i^ captured by 
questionnaire data in response to the following multiple choice ^Item embedded 
among others related to course components. The responses j^re scaled from 
+2 ,to -2. ^ • • ' ^ ■ ' ^ 



The CAPS^problems were: 
• a. very detrimental 
^ b. detrimental 

c- neutral , 

d. ^ beneficial 

very'benef Iclal - ^ 



(Assigned Scale) 
(-2) . . 

i 0) 

(+1) ^ V 

(+2) 
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The scaled data for end~of-semester frequencies were compared to the 
data for other course components. All items had the same response choices 
listed above for the CAPS item. The scaled da'ta resulted in the following 
rankings: 



'Fal 1 Semester 



- Sprin^g Semester 



Rank 


Component • 


Rank 


Component ... 


• Ranij 




Workbook 




Workbook 


1.80 


2 


Lecture 


1.25 ' 


Lecture 


1.10 


3/ 


■ PLATO (luizzef, 


hl"8 . 


Protl em Sess ion 


.80 


k ' 


CAPS ' ' 


1.04 


PLATO Quizzes 


r .76 


5 


Problem Session 


.88 


CAPS 




. 6 


Primary Textbook ' 


" .07 


.Primary Textbook 


,'-.90 



In ah open-ended quest l^bn,' students were asked Tb^give their reactions 
to the course components. Students'*' responses i^ndicated that the most dis- 
satisfaetioQ with the CAPS sy^stem stems from the inconvenience locating 
a|( available terminal at a copven ijent ,^^y^e^. At least tvS?c> students rented 



terminals for home-use since they' are avai'labVe at a nomi'nal cost. 



Ijer-^iir strategy:^^ a 1 lowing only one item at a time to be available 
for a studerft •was , designed 'so that 'students would keep^ current with the^ 
lecture topic/ student's wj^o opted for fewer — |)ut longer~session*s had some 
cUfffCulfy in relating CAPS problems* to cjass discussions. Periodic in- 
spectffon of ^tudSnt performance data indicated that some students were 3 " ^ 
topics behind class discussions. Consequently, instructors^ did not discuss 
related CAPS problems during the lectures. . 
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* Most student reaction to CAPS, however, was positive. While some 
sfudents 'did.ngt * keep 'curreQt , ' most students reviewed several items priqr^ 
to the final' exams. All students successful \y 'copnpleted aM CA'PS i tents' ^1 • 
prior to.„tbe f in^l exam, for boXfe semesters. One particular student reaction \ 
deserves'mention because it -captures one intent of. CAPS: "The CAPS prbblems 
are useful. In areas I am confi dent, : they lend support to cbnfidence. In 
problem areas, they clar i fy 'methodplogy (Fall semester).*' 
Summary ■ " ' 

Although we have employed an interact^ive computer system normally 
'used for coding and executing programs,*: the CAPS system can betused for 
individual problem sol vi^ng, . wi th computer assistance if necessary, and cap 

.^.e \i§g^tf6j^lndlv.i^i.L|^/l,^^ sessions v^ith the instructor if desired. 

k i ^ * 

Although intlividual problems were not always integrat^ed into the lectures-- 

^r wa% that the i ntent--the -CAPS system, by providing concrete examples 

of concepts taught in class, was an int^egral .part of these sections o/ the 
introductory stat istics. class. ' ^ 

During'the Fall, 1977 semester, the CAPS system was modified so that 
the item sUtrout ines 'formed the basis of a problem generatiW system that 

>»as used without student contact with the computer. The instructor wa^$. able 
^^o produce individual problems for hi s "students with separate answers for his 

'own use. Thdse minor modifications required that the instructor interact ^ 
with the computer while the problems were being produced. We speculate that 
the CAPS system could, be modified further so that the items could be run in 
•batch^ mode rather than just interactively* ^ ^ \ ' 



Computer Assisted Problem Solving 

The preseli^t format of CAPS could be used by other s'ubject matter dis- 
cipllnes which normally assign problem setk emphasizing appfication. Such 
conter^t areas might include accounting, physics, and chemistry. 
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